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1 Problem 

We study differentiability of the multidimensional scaling loss function rStress ((???)), defined 
as 


a r {x) := ^2wi(Si - ( x'Aix ) r ) 2 (1) 

i =1 

for some r > 0. Here the uy are positive weights and the 8i are positive dissimilarities. The 
matrices Ai are positive semi-definite, and the quantities x'AiX are squared distances. 

Clearly if x'AiX > 0 for all % the loss function is differentiable. De Leeuw (1984) proves 
directional differentiability for r — f and he shows that at a local minimum we generally 
have x'A^x > 0. We investigate if and how this results generalizes to a r . 


2 Directional Derivatives 


Define the directional derivative 


da r (x,y) 


lim Qrjx + ey) - cr r (x) 

e4-0 6 


1 



For our computations we need 


I+(x) := {i | x'AiX > 0}, 
Iq(x) := {i | x'AiX = 0}. 


Then 


a r (x + ey) — a r (x) 


= -4 r Wi(6i - ( x , A i x) r )(x , A i x) r l y’AiX 
iei+ 

- 2e 2r ~ 1 J2 u> i Si(y'A i y) r + e 4r_1 J2 Wj(?/A?/) 2r + 
ieio ieio 


o(e) 


and thus 


da r (x,y ) 


f-4r EILi - (x'Aix) r )(x'AixY^y'A { x if r > |, 

| -4r Eie/+ - (x , A^) r )(a: , A^) r “ 1 2/ / >li^ - 2 Eie/ 0 w A{y' A iy) r if r = |, 
[+oo if r < 


3 Results 

From our computations we derive the following results. 

Theorem 1: If r > | then oy is differentiable at x. If ay has a local minimum at x then 

n n 

^2wiSi(x'Aix)^ 1 AiX = ^Wiix' Aix) 2r ~ l AiX. 

i —1 i =1 

Theorem 2: If r = | then ay is directionally differentiable at x in every direction y. If o> 
has a local minimum at x then 

X! = JZ -iUj(x / Ax) 2r_1 A^- 

iG/+(s) i£l+(x) 


and Iq(x) = 0. 

Theorem 3: If r < | then ay is directionally differentiable only in those directions y with 
y'A t y = 0 for all i e Iq(x). 


Thus for r = 4 we have non-zero distances and differentiability at local minima, for r > 
ssi 

even directionally differentiable at points with zero distances. 


it is quite possible that local minima with zero distances exist, and for r > ’ rStress is not 
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4 Local Maximum 

We can also generalize a result of De Leeuw (1993) to rStress. 

Theorem 4: a r has a local maximum at x if and only if x — 0. 

Proof: If x = 0 then 

f n 1 n j 

a r (x + ey) - a r (x) = -2e 2r | Y,w i 6 i (y'Ay) r - -e 2r ^ w i {y'A i y) 2r ^ . 

It follows that if 

1 2r< £?=i wMy'AyY 

2 e - Z'UMy’Aivr 

we have a(x + ey) — a(x) < 0. So, although a r may not even directionally differentiable at 
x — 0, it does decrease in all directions and is thus a local minimum. 

Converse, suppose a r has a local maximum at x ^ 0. Then 

n n n 

a r (ex ) = "^2 w i$i —2 6^2 Wi8i(x'Ax) r + 9 2 ^ Wi(x'Aix ) 2r , 

i =1 i =1 i =1 

with 9 := e 2r . Thus oy is a convex quadratic in 9 and it cannot have a local maximum on the 
ray through x. QED $$ 

5 NEWS 

001 01/14/16 - First upload 

002 01/15/16 - Added local maximum result 

003 02/08/16 - Corrected some typos 
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